Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Design

Sparse training is one of the promising techniques to reduce computational cost DNNs while retaining high accuracy. In particular, N:M fine-grained structured sparsity, where only N out consecutive M elements can be nonzero, has attracted attention due its hardware-friendly pattern and capability achieving a sparse ratio. However, potential accelerate DNN not been fully exploited, there lack efficient hardware supporting training. To tackle these challenges, this paper presents computation-efficient scheme for using algorithm, architecture, dataflow co-design. At algorithm level, bidirectional weight pruning method, dubbed BDWP, proposed leverage sparsity weights during both forward backward passes training, which significantly maintaining model architecture accelerator namely SAT, developed neatly support regular dense operations operations. multiple optimization methods ranging from interleave mapping, pre-generation weights, offline scheduling, are boost efficiency SAT. Finally, effectiveness our evaluated on Xilinx VCU1525 FPGA card various models datasets. Experimental results show SAT with BDWP method under 2:8 ratio achieves an average speedup 1.75x over that accompanied by negligible accuracy loss 0.56% average. Furthermore, improves throughput 2.97~25.22x energy 1.36~3.58x prior FPGA-based accelerators.